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experts in impact assessment regarding the impact assessment system/ 

3, To revise and complete the impact assessment system for subsequent 
•operation by the Research ai^d Development Section Staff, 
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1.. ^Reviewed and revised the impact assessment system, 

2. *Met with Research and 'Development Section Staff to gain inplft on the 

impact* assessment system. > ^ 
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4, Revised and completed the impact assessment system, 1 ' 
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resources spent for funded "projects improved the programs identified by the 
Research and Development Section, 
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tn Jur\e/1982 ' . ' 



ERLC 



■ // 




TABLE OF CONTENTS 



K Introduqtion^'and Overview : 1 

•A. Phase I - 1978-79 1 

' . B. Phase II.- T979.80; '2 I 

C. Phase III - 19,80-81 .......^.l 3 

'D. Phase IV - 1981-82... 3 

II. , Resources Utilized ^ , 5 

. ; ; A. Expenditure of Foods .5 

. B. ' Paid Parttci'pants. \ 5 

C. Resources.; 6 

lU. Accdmplishnients and ^Major ^ActiVi ties. T 7 > 

' \ A. Atta.inment of Project Objedtives.! 7 

' B^ , Project- Activities;......; 7 

Relatljd' Activities ' 8 

. D. Problems and Conp-erns... ' 9 

1 Statement of y Impact, ' 9, 

IV, Recommendations 10 

• *^ ^ » 

Reference List 12 

Appehdi)^ A 

, (A Review antf Analyses of Fiscal Year 1982 Continuation 
, • * 'Proposals' Statement's of Impact) • ^ ■ , 

Appendix B.^ ;v .i, , 55 

CSximmary of AERA cjDnference attendance) ^ ' 




I. INTRODUCTION AND OVERVIEW • . ^ • 

• The primary purpose of funded vocational education research^and ^ 
de.v.elopment activities is to. improve vocational education programs. To 
determine .if these funded efforts make a ^c^ifference, we must identify the 
changes (skills, knowledge, attitudes, interests, values, perceptfons) 
that occ3(jr ajpfng specific aud'ienCes. I^es^arch and development activities 

^are expensive,. Therefore, it is critical to assess their impact (conse-* 
quences) as a basis for.mak-tng bette^ judgmenlfs about future activities, 
A. Phase I •1978-79 • . " ' 

^Public Law 9.4-482 mandated that contracts for Research and Develop- 
ment ,in VolatioBdl Education would not be"* made unless the applicant could 
"demonstrate a reasonable probabil ity". that the.contract- would .result in 
improved teacl^ing techniques or curriculum materials that would be used 

, in a "substantial number of classrooms or othep/l earning situations within 
five ye^ns. after t'erpilnatiOn of such contracts" (Federal Register, 1977). 
As originally conceptualized, 'the purpose of Phase T of the Impact Proj-ect 
was to identify and develop procedures for .complying with the impact re- 
quirement of Public Law^4,-482. 

To that end,"'the Impact Project addressed three problem areas during 
Phase I activities: (1) how-to define impact, (2) how to, assess impact, 
and (3) how to show relationships between project activities and changes 
in vocational education teaching-learning situations. The project also 
addressed two subsidiary problems: (1) how t? predict the probability 

of impact, and (2J how to manage on-going. contracts to increase impa-ct 

7 ^ 

probability. 



Phasfe I (rFY 1978,) activitiesj^included: '^1) review tff literature 
related to ,th^robTan areas, (2) interviews of indivldualsl'who had 
experience and expertise In the problem areas, and '(3) analysis of the 
process of impact^ by foUowi^ng several programs of related projects. 
The majoV activity fdr Phase .i activities focused on analyzing, 
a case study approach, prQgram^* of related research and development 
projects. It was decided to select two cases for "top-down" analysis ^and 
two cases for "bottorntup" analysis. The pl^dject staff referred to these 
typjes of retrospective analysis as "tracking*" It was anticipated that 
two types of tracking-would produce different ins'ights about impact. , The^, 
top-down "tra.cking" included "jminois Occupational Curriculum' Project" , 
and "minOis Network of Exemplary Occupational Programs for Handicapped 
and Disadvantaged Students." - Bottom-u|) "trackinglL included t'he "ininois 
Projects in Horticulture" and "minois Career Education Projects at the 
Awareness Level." * ' - ^ ^ 

B> Phase II - 1979^80^ ; ^ 

These case study efforts were continued during- Phase II (FY 1980) 
.activities. In addition, P+^ase II activities included tw6 other case 
studies, a field stKJdy, and' development of a model for art "Impact 
Assessment System for the Illinois State Board of Education/Department 
of Adult, Vocational and Technical Education/Research'and Developjnent 
Section." The two n*ew-cases selected for study were the "Occupational"' 
Survival Skills Project" and '^'Two^ll inois School Districts with 
Innovative Vocational Education Programs." "The field study focused o/i 
"Predicting Impact of Research and Development Projects in Vocational 
and Technical Education."' It Is to the last activity outl ined for 



Phase II, development of a model for an "Impact Assessment Sys^m for the 
Illinois State Board of Education/Research and Development Section,'" to 
which Phase I II -project activities were focused, 
C, Phase III 1980--31 ' • . 

The" main objective for Phase III (fY 15"81) was to review, further 
Conceptualize, and refine the iaipact assessment ;5/stem for vocational- 
education research and devalbpment efforts proposed during FY 1980, To 

achieve this objective the following procedures were completed: 

> 

1, Revriey/ed and further conceptualized the proposed . 
impact asse'ssment system developed during FY 1980, * , 

2. Studied tl^e organizational structure and management 
of the -Research and Development Section regarding 
program improvement efforts. , ^ 



3^ Identified the perceptions of project directors and 
Research and Development Section Staff regarding 
impact • ^ 

4, Conceptual fzed a comprehensive impact assessment 
. system, , . " 

5, Identified component's vital to a comprehensive impact 
assessment system* ^ - > 

6* ' Developed a workiTig document of the\impact assessment , 
system to guide the f4iia.I phase of the project in 
FY 1982. ^ ' 

D. gbase IV 1981-82 ^^ , , 

The mairi purpose of Phase IV (FY 1982) was to refine and complete the 

'impact assessment system for subsequent operation by the Research and 

Development Section Staff. To a'ccomplish this purpose, three major 

objectives were pursued. The first was to review and refine the components 

of the Impact ^ssessmfent system. The. second was^to gain input from the 

Research and Development Section Staff and experts in impact assessment 



4^ 



i*ega(rdihg the impact assessment system. The thifd was to revise and 
complete the impact assessment system for subsequent operation by the 
Research and Development SecWon Staff. % ' 
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II. RESOURCES UTI LI-ZED 

M Expenditure of Funds 

.The project was completed within the financial limits established in 
the budget of the funding agreement. Actual expenditures claimed were less 
than expected in some cases. ' ^ ^ 

B. Paid Participants in Project 

This project was an integral part of the Department of Vocational and 
Technical Education'at the University of Illinois. Therefore, several 
faculty members were involved. Paid participants in the "project were: 

K Project Director - Dr,. Rupert N,^ Evans, Professor" 

2. Co-PV-oject Director - Dr. Earl B. Russell, Associate Professor , ' 

3. Principal Investigator -^Dr. Wynette S. Barnard, Visiting ' 
Assistant Proceffor ^ ^ «» ' 

4. Staff Member - Dr. Carol S. Sanders, Visiting Assistant Professor 

5. Graduate Assistant - Eva ^. Coffey 

In addition, four paid consultants with interest and expertise on the 
topic of impact assessment were utili/ed'to critique the impact assessment 
design and offer advice for improvements. Paid consultants for the project 
were: 

1. Dr. Kay A. Adams, Coordinator of Evaluation, National Center -for 
Research in Vocational Education, Ohio State University 

i. Dr. Henry M. Brickell, President, Policy Studies in Education, 
New York, New York 

3, Dr. Mary B. Malone, Professor, Graduate School of Education, 
Rutgers University, New Brunswick, New Jersey 

4. ' Dr./iRonald D*. McCagey Director , Vocational -Technical Education 

Consortium of Stated Southern Association of Colleges and 
Schools, Atlanta, /Georgia* • 




Resources , ^ ' 

Project staff for this effort were housed in the Department of Vocational 
and Technical Education fa,cilities within the Education Building on the 
University of Illinois campus. Various University facilities were , available 
to the project staff which greatly faailitatied the actoijpl ishmentvof project ' 
activities^v The resources of particular val.ue to this project included: 



ffice of Vocational Education Research; Tim Wentling, Director 



University of Illinois Library (catalogued collection irt excess 
of 5% million volumes, and ah uncatalogued ^call ection of 3% 
million* volumes) * 

Bureau of Educational Research; Steven Asher, Director 

Center of Instructional Resources and Curriculum Evaluation; 
Robert Stake, Director* ' ^ ^ 
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• ; ^ III. ACCOMPLISHMENTS AND* MAJOR ACTIVITIES 

k. Attainment of Project Objectives 

The followina objectives were accompl ished'^durihg the project: 

1. To review and refine the impact, assessment System, 

^ 2. To sain input from the Research and Development Section 
Staff and experts in impa'ct assessment regarding the 
impact assessment system. 

•3, To revise and complete the. impact assessment system for 

subsequent operation by the Research and Development \ 
Section Staff* ' * • * 

* ji- ' 

B. Project Activities 

The following activities wl^ich. are rela-ted to each objective were 
completed^ during the project: 

Objective 1— To review and refine the impact assessriient system 

1.1 Continued to review documents related to impact assessment to 
gain ideas for the refinement of the impact assessment system.. 

1.2 Spent time in Springfield reviewing continuation proposals for 
impact related information. 

1.3 Analyzed impact statements from continuation proposals to 
determinre if proposal writers understand the concept of 
impact and the new guidelines for writing impact statei|ents. 

1*4 Wrote a report summarizing the analysis of impact statements 
(See Appendix A' for a copy of the report.) 

1,5 JJeceived and reviewed feedback from the Research and Development 
Section Staff on the impact assessment system proposed in 
FY 1980-81. V 

♦ 

Met with Jim Pearsol , Evaluation^ Special ist (Center for 
Instructional Research and Curric'ulOm Evaluation, University 
of Illinois) to gain input on the propos?ed impact assessment 
system. ^ ^ - . ^ 

1.7 Met with Bilt^Hull (The National Center for Research in 
Vocational Education, Ohio State University) to gain input 
on the proposed impact assessment system. 



i3 
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K8 Revised J:he'- impact assessment ''syst'em^ ; ' ./ . V 

Objective 2--To gain input from the Research and Development Section Staff 
anjl experts in impact assessment regarding 'the impact assessment system, 
* • 

2.1 Submitted the revised impact assessment system-^to the Research 
and Development Section, ^ 

2.2 Met with Peter Seidman and John Washburn to discuss' the revised 
impact assessment system' and how current problems. and issues 
within DAVTE migKt affect the ifoplementation of the system. 
Revisions- of project objectives 'wer.e also discussed, 

./ ^ , ^ ^ • ; 

2.3 Developed and presented "Impact Assessment of Research and 
Development 'Program Improvement Efforts" paper at the American 
Vocational Association Convention as^part of an AVERA research 
symposium. The paper was one of foi/r presented at "Linj(;ages 
to Promote Program Improvement" symposium, 

' . : . > 

2.4 Discussed the impact assessment system with numerous persons at 
the AVA convention, gaining additional input into the refinement 

of the system, 

» 

2.5 Sent the impact assessment system to four external consultants 
for their review and critique, 

2.6 Obtained feedback from the consultants for further refinement of 
the impact assessment system. 

Objective 3— To revise and, complete the impact assessment system for subse- 
quent operation by the Research and Development Section Staff, 

3.1 Reviewed feedback from the Research and Development Section Staffs 
consultants and other resourcen>ersons, ^ 

3.2^ Met with Peter Seidman, Contract Administrator, to review feedback 
and to discus^ final revision, 

3.3 Compl eted the In/pact, gssessment system'. 'This document, entitled 
Impact Assessm/it Design , is available from the Department of 
Adult, Vocational and Technical Education, Illinois State Board 
of Education. ^ 



C. Related Activities » 

In addition to the project activities described, Projec^ Impact Staff 
met W|ith the staff of the project "A Study, of the Non-traditional and Social 
Impact of Vocational Education on Individuals in Illinois", to initiate 
collaboration efforts between- the two projects. Collaborative efforts 
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throughout'' the year involved several meetings which prom'oled an ajf/areness ^ 

and understanding of the purpose of each project and methods utilized io tjie 

^projects. "Two Project Impact staff members also attended the workshop on the 

longitudinal case study project. A second related activity was attendance at 

the American Education Research Association annual meeting (See Appendi)^ B). 
D. Problems and Concerns 

After meeting with Peter Seidman, Contract Administrator, and John 
Washburn, Manager of R & D Section io D/\yTE/ISBE, it was decided that 
because of some uncertainty as to when the impact assessment system could 
be implemented, some of the. proposed project activities were inappropriate. 
Those deemed inappropriate were related to the ^velopment of staff 
development materials and the provision of in-service training. Therefore, 
additional time was spent on the refinement of the impact assessment 
system in lieu of those activities. 
E> 'Statement gf Impact 

By implementing the system developed and refined by the project, the 
Research and Development Section Staff can provide assistance to project 
directors and project proposal reviewers to (1) become aware of the need 
for impact assessment, (2) become Ijinowl edgeabl e cf^how to facilitate impact 
assessment during proposal development and project activities, and (3) 
understand the conceptual framework of short and long term impact^ assess- 
ment and the relationship between projects and programs . Thus, the ^ 
overall impact upon vocational education will be to indicate if the 
'resources spent for research and development projects improved the 
programs which had been targeted by the Research and Development Section. 
The system will provide a strengthened approach to sharing the benefits 
of vocational education research and development with constituent groups 
and. policy makers. 



IV.-^ RECOMMENDATIONS ' 



The fol Towing recommlndations to the R & D Section, DAVtE/ISBE, are 

based on the refined impact assessment system. The recommendations are 

* *• ^ ' 

divided into thr,ee categories: Implementation of the Impact Assessment 

*• * 

System, changes in the .Statement of Impact, and^ Progress Reports, 
Implementation o.f the Impact Assessment Design . ^ 

Immediate 

1. Assess the short term itnpact of a sample of#*^ 
completed projects , 

2. Assess the long term impact of a sample of 
' projects . 

Within five years 

1. Continue to assess the short term impact oT a 
sample of projects each year, 

( . 2, Assess the long term impact of programs , (to^ 

replace long term project impact assessment) 

S / 

Within seven years 

1, Continue fo assess the short term impact of 

' -projects and the long term impact of programs 

^ ■ I 

2, Expand the impact assessment analysis to include 
^all classifications of impact. 

Statement of Impact 

It is recommended that Section 5.6 (Statement of Impact) of the RFP 
Guidelines be changed to the following: , • 
5,6 Statements of Impact , - 

The statements of impact are statements describing *the outcomes of the 
project goal(s) (see subsection 5,1), Short term impact statements reflect 
the intended consequences of the project within a year of the termination of 
funding* Long term impact statements reflect the intended consequences of 
the project goal(s) one to five years after the termination o^^unding. 
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^ In (describing the intended consequences of the project goalCs). 

the following elements must be, included: 

^» 

t'The target populiation(s) expected to be affected 
• The prec*is-e nature of intended consequences (e,g,,xhange :* 
- in specific skills, knowledge, 'attitudes, interests, 'values,. 
perceptions\) • v ^ ' ' ^ \ ' * 

Example' Impact Statements - 

♦ 

The short term impact of this project will be that 
•teachers usin^ the entrepreneurship educatiQrf*^ur- k, 
riculum materials will improve their abilit/ to • - 

teach entrepreneurial skills, / 

The l^ng term impact will be that students of ^ 
teachers using the entrepreneurship education 
curriculum materials will develop -entrepreneurial 
skiTls and be able to apply these skills in their 
jobs and everyday life. 

Progress Reports 

It is recommended that Section. 9 (Statement of Impact) be changed to read 
"Evidence of Progress Toward- Impact,", , This will facil itate- the collection of 
impact data during the funded life of a. project. 
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INTRODUCTION . 



The primary purpose of funded vocational education research and 
development activities is to improve vocational education programs. 
To determine if these funded efforts ntake a difference, we must 
identify the changes that occur among specific audiences as a con- 
sequence of project activities. 

The first step in determining if funded efforts make a difference 
is to determine if conscious efforts are made in the planning process 
to identify intended consequences of projects and programs. The total 
process may be referred to as impact assessment. 

In Assessing the Impact of Vocational Education Research and 

Development on Vocational Education Programs > (Project Impact), Final 

Report /Phase III, Impact, intended impact and impact assessment are 

defined ^as follows: ^ 

"Impact is the consequences, positive or negative of 
funded vocc^tional education research and development, 
program improvement projects and programs." (p. 62) • 
Impact is further characterized as being short term or 
long term; intended or unintended; direct or indirect. 

•'Intended impact occurs when the consequences of projects 

and programs wer^e planned for and/or expects." --(in^^)- 

Impact assessment is "a process which starts with the 
developpient of a proposal specifying the impact intended 
as a result of project activities. The process continues 
throughout the funded project effort as impact statements 
are updated to accommodate changes in. project activities • 
and continues beyond funded efforts as final project 
outputs are disiseminated." (pp. 20-21) 
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since 1979, proposal writers in Illinois seeking funds for 
research and development in vocational education have been required 
to! specify the-intended inipact(s) which will result . from proposed 
project activities. The intended impacts are identified in impact 
statements include;! in proposals'for^ funding, based upon project 
goals that Indicate tmt intended consequences of the project activities 
upon a target audienq^^, (A ssessing the Impact of Vocational 
Education Research and Development on Vocational Educatiorri^ograms, . 
Project Impact, Final Report/Phase III) / 

The extent to which--pnoposal writers have specified the intended 

impacts to result from their proposed project activities is described 

In Assessing the Impact of Vocational Education Research and Development 

on Vocational Education Programs, (Project Impact), Final Report/Phase III 

which states: • * - , . - 

"Guidelines that 'accompanied the FY 1979 and FY 1980 

Requests for Proposals reflected the Mandate of Public 

Law 94-482 pertaining 'to asses^ent of impact. Proposal 

writers did not understand what was meant by impact and % 

all that it 'implied. Proposal requirements regarding 

Impact confused and frustrated. proposal writers; the 

proposals did not contain all* that was required pertaining 

to impact", (p. 13) 

After reviewing FY U79 and FY T^QSO proposals. Project Impact staff 

« 

concluded that: . - \ 

— ^'Content of required intfended impact statements varied 
greatly With most addressing the extentof dissemination 
and not the intended consequences of a project", (p. 13). 

To aid proposal writers in Illinois to differentiate between dis- ^ 

semination and impact and to understand that dissemination is a necessary 

part of but not syhonomou's' with impact, ^Project Impact staff recommended 

that the two concepts be addressed"" separately in proposals. 
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Recommendations of Project Impact staff regarding separate 

sections fo r addressing i mpact and dissemination in proposals for 

funding were written and presented to the Illinois State Board of 

Education, Department of AduH, Vocational and TechnicaV Education, 

Research and Development Section. The recommendations were incor- 

porated in^to the Guidelines and Specifications for Preparing FY 1982 

Continuation Proposals . Subsection 5,6 of these Guidelines requires 

proposal writers to inclu4e in their proposals a "statement of impact". 

Specifically, the requirements of subsection 5,6 are as follows: 

"The statement of impact is a narrative description 
of the outcome(s) of the project goal(s) (see sub- 
section 5.1). The impact statement reflects, the 
intended consequences of the project goal(s) upon 
termination of funding. 

In discussing the intended consequences of the project V 
goal(s*) the following elements must be included: 

- The target population(s) expected to be affected. 

- The precise ^nature of intended consequences, (e.g. 
change in specific skills, knowledge, attitudes, 

interests, values, perceptions).*' ^ 

Subsection 5.1 of the Guidelines is the introduction in which the 
propos'al v/riter presents a rationale for the project, summarizes project 
goals, and states how achievement of the goal(s) will benefit vocational 
education within and outside the state. This subsection provides the 
basis for developing the statement, of impact in Subsection 5.6. 

• ' To determine if changes in the FY 1982 guidelines for proposal 
writers were effective' fft helping the writers to improve ^their state- 
ments of Intended impact. Project Impact staff assumed responsibility 
for reviewing FY 1982 proposals. This responsibil ity as stated in 
Project Impact Final Report/phase III is. as follows: 



"The intended Impact statements in FY 1982 proposals will 
be analyzed early in Phase IV to determine if proposal 
writers understood what was required and to determine if 
further clarification is needed"^ (p. 14) 

The following report summarizes the findings of Project Impact 

staff as a result of fulfilling this responsibility. 



GUIDING QUES'TIONS,and REVIEW PROCEDURES 
Guiding Questions ^ 

Based on the requirements for completing the subsection on impact in 
Guidelines and Specifications for Preparing FY 1982 Continuation 
Proposals the following questions v/ere developed for the review:* , 
K Did proposal writers follow the' procedures as outlined in 
subsection 5.6' Statement of Impact by: 

IJ Identifying "Statement of Impact" in/as subsection 5»6 

in their proposals? * ' 

T,2 Describing in the "Statement of Impact" the outcomes of 
the project ^oals? (Is it evident that subsection 5 J , 
Introduction,' was used as the basis for writing subsection 
) 5,6 as specif! ed^in the gu7delines?y 
\.Z Describing the target populations to^be affected? 
2» Did proposal writers include references to dissemination in the 

project proposals* statements of impact? 
General Review ^Procedures • * ' - 

Project Impact staff reyiewed sixty of the seventy-one project " 
, proposal Si' funded for fiscal. yegr.J 982 by the Illinois State Board of 
Education, Department of Adult, Vocational and Technical Education, 
Research and Development Section, The list of proposals reviewed is 
Included in Appendix A-1 

All proposals funded were ta'be reviewed exce.pt Assessing, tlie 
Impact of VocatlonaT Education Research and Development on Vocational, 
Education Programs and Vocational Education Program Dissemination 
Phase III, the two projects with which Project Impact staff are ^ 



involved. During the da^ta -collection period; however, nine of the 
'iixty-nine proposals remaining were not available for review. A list 
of the nine proposals not included in tlie review is in Appendix A-2, 

During data collection, Sections 3 and 5 of the sixty proposals 
were reviewed. Section 3 "Proposal Abstract" and subsections of 
Section 5 '^'Narrative Description o^ the Project" were photocopied for 
more thorough review. The subsections of Section 5 which-were photo- 
copied included: 5,1 Introduction, 5.5 Dissemination Planning Outline, 
5.6 Statement of Impact and 5.7 Time, Sequence 'Chart. The subsections 
were not in all cases labeled precisely as identified here nor did all 
proposals contain all the subsections identified for photocopying. The 
fact^that some subsections were not labeled precisely did not cause 
problems in the review because if present, the subsections were *. 
identifiable. However, some propQsals did not contain all the proposal 
subsections identified for review. ^ 
Specific Review Procedures^ . ^ 

Question 1.1: To determine if proposal writers followed the 
guidelines for ^identifying the subsection on impact correctly, each 
^jroposal's subsection on impact was' reviewed to see how it was labeled. 

Question 1,2: To determine if proposal .writers related the 

statjement of impact to the ♦project goal(s), the reviewers'used two 

procedures. The first plhocedure involved identifying the project 

goal{s) tn subsection 5.1 and analyzing the statement of impact to 

determine if the goal was reflected in the statement. For example. 

Project Impact in subsection 5.1 identifies its goal as follows*: 

"The goal of the final phas6 of this project is. to 
refine and. implement a system for assessing .the short' 
and long term impact of vocational education research and 
development* in vocational education pr'ogrems*" 



Subsection 5.6 Statement of Impact states: "By implementing the 

system developed and refin)sd by the project, the Research and 

Development Section can provide assistance to project directors and 

project reviewers tp (1) become aware of the need for impact assess- 

raent, (2) become knowledgable.of how to facilita1?e impact assessment 

during proposal development and project activities, and (3) understand 

« 

the conceptual framework of short and long term assessment and the . 
relationship between projects and programs." ^ ^ 

The relationship between the project goal and the.Statement of 
Impact is easily identifiable in this example. The goal of the project 
is to ''^refine and implement a system. . The impact statement reflects 
the consequences 'Of utilization of the system. Unfortunately, the re- 
lationship between subsections 5.1 and 5-.6 are not always so clear. In 
some project proposals a goal(s) was not identified and/or not apparent 
to the^ reviewers. In these cases, a second and different procedure was 
used for review. 

, For project proposals in which a goal was not identified or not 
apparent to the reviewers the. objectives of the project were used as 
as indirect way of determining if\he statement of impact reflected 
the goal(s) of tj)e project. The assumption was made that the project 
objectives were closely related to and an outgrowth of the project's 
gbal{s) whether or not the goal was. explicitly stated. 

To determine the extent to which the statement of impact reflected 
Intended consequences of the project goal(s) the following criteria 
were used: 

7 
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1. Impact -statenitent describtes most outcomes of .the project 
goa1(s} or objectives. ■ ' 

2. Impact statement describes some outcomes gf the project 
goal(s) or objectives* f ♦ 

3. Impact statement does not describe outcomes of the project 
goal(s) or objectives. . 

4. Impact statement not availabfe for' review* 

y 

It should be brought to the readers' attention that review .of this 
component of the statement of impact because it required some judgment 
on the part of the reviewers may be influenced by ^e reviewers' 
understanding of and ability to interpret the relationship befween 
subsection 5.6 and goals/objectives. Also a number of proposals' , . 
impact statements, though reflecting the intended consequences of the 
project goal(s), did so without the speoificity the guidelines require. 
The guidelines request "the precise nature of intended consequences 
(e.'g. change In specific skills, .knowledge, attitudes, interests,, etc.X" 
Many proposals did not reach this level of precision,^ In^ese 
proposals, proposal writers tended to center on the problems motivating 
them to write the proposals and\the needs these problems engender. They 
fatled to identify the changes in skills, knowledge, attitudes, interests, 
etc^, which they intend to^^be consequences of their projects' efforts. 

Question 1.3^ To determine the extent to 'which proposal writers 
tdentlffed the target popuJatlon(s) expected to be, affected by their 
proposal, the populatlon(s) identified in the project goal(s), objectives 
and/or dissemination planning outline was compared with the target popu- 
latlon(s) Identified in the impact statement. 
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Question 2*0* The proposals' subsections on impact wei«e , 
reviewed to determine the extent of iric'Tusion of dissemination 
references In the Impact statement by. dfeterminihg if such references' 
were ma'de and if mdde', were they e;<p1icij: or implied?. 

JTreference was considered to be explicft If the word;- : 
"d lis emi nation" was.used^'n the impact .statement or {i'f k' specific 
dissemination activity" (activities) was identified in tjfie impact 
'statement. A reference to dissemination was considered to be ^ 
implied if a statement led one to assume sbrrte dissemination would 
be necessary for the act^ion in the statement to take place. 

A determination of "no reference to' dissemination" was made 
when the content of the impact statement could not be categorized 
using either of the above two criteria. 
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FINDINGS OF THE REVIEW 



This section presents the findings of the review of fi.scaTyear 1982 
continuation proposals for treatment of requirements relating to^ impact. 
The questions which guided the review serves as t^ basis for structuring 



Question 1 ,1 

Did proposal writers follow the new procedures as outlined in sub- 
section 5.6 - Statement of Impact, in Guidelines and Proceduv-es Jor 
Preparing FY 1982 Continuation Proposals by identifying "Statement 
of Impact" in/as subsection 5,6 in their proposals? 

Figure 1 provides the results for the sixty proposals reviewed. Of 

the sixty proposals reviewed, twenty-two (.36,7%) labeled Statement of 

Impact in/as Subsection 5.6. Writers of five (8.3^) of the propo^sals 

labeled Statement of Impact in/as Subsection 5.5. In Request for Program 

Improvement Proposals in Vocational Education 1950 , Statement of Impact 

was to be labeled as Subsection 5.5 therefore it is possible,- even 1 ikely 

that these proposal writers used the guidelines contained in that pub-- 



licatlon when preparing, their FY 1982 proposals. 

Twenty-three (38.3^> of the proposal writers labeled the subsection 
"Statemetit of Impact" by using a number(s) 6thfir than 5.5 or 5.6, by using 
a topical heading only or by using a letter of the alphabet. For ten 
06.7%) of the proposals there was not a subsection - "Statement of 
Impact" available for review.^ (See Appendix B for subsection identifiers 
used for identifying subsection on impact for Individual project 
proposal s)* , ' ' 

It is important to point to the fact that in most proposals, 
regardless of how the subjection "Statement of Impact"was labeled, it 
appeared in the proper place according to the topical sequence outlined 
In the guidelines. ^ * ^ 
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Statewent of Impact identified 
as Subsection 5.^6 

Statement of Impact identified 
as Subsection 5.5 

Statement of Impact identified 
by a number (s) other than 
5.5 or 5.6 




Statement of Impact identified 
by a Topical Heading Only 

Statement of Irxpact identified 
by a letter in the alphabet 

Statement of /Mnpact subsect 
not availabli for review 
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Figure 1. Classification of Subsection on Impact in FY 1982 continuation proposals 



Question 1,2 



\ 



Did proposal writers follow the procedures ai outlined in subsection 
5.6 i Statement of Impact in Guidelines and Procedures for Pr^eparing 
FY 1982 Continuation Proposals by describing In the Statement of 
Impact the outcomes of the project's goal(s). (Is.it evident that 
subsection 5.1 was used as the basis for writing subsectioh 5.6 as 
specified jn the guidelines?) . ^ 

Figure 2 summarizes the findings of this review. Thirty-four (5Q.7%) 
of the project proposal s' statemen ts of impact reflect most of the intended 
outcomes of project goal s/objecti yes. ' 

Sixteen (26.6%) of the project proposals' statements of impact reflect 
some outcomes of project goals/objectives. Where the determination was 
made that the statement of impact reflected "some o^he intended outcomes 
of the project's "goals/objectives," it was because several intended out- 
comes Identifieci In the goals/objectives' were not incorporated into the 

statement of* impact, -7 
♦ » 

In no- project proposal did the statement of impact, when available 

* ■ ' ^ 

for .review, faiV to describe at least some of the intended outcomes 
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identified In project goals/objectives; however, ten (16,7%) of the 
project proposals' statements of impact were not available for review. 
For these ten proposals, the section "Expected Contribution or Potential 
Impart oirTocationaT€ducat1~o7T**"in~tfre"projedts^ abstracts was, tn most 
cases, descriptive of the outcomes of goals/objectives. In these ten 
proposals, the abstract contained the only statement of impact since 
these proposals did not contain a subsection on impact in Section 5 - 
Narrative Description of the Project in their proposals. 



statement of Impact describes 
most outcomes of project 
goals/objectives ^, 

Statement of Impact describes 
some outcomes of project 
goa1s/ol}ject1ves 



Statement of Impact does not 
describe outcomes of project 
goals/objectives c 



V Statement of Impact not 
available for review 



56. 7X 




(Number of Proposals) 



Figure 2. . Degree to which the Statement of Impact reflects ilitended consequences of project goals/objectives. 



(See Appendix C for the extent to which the statement of impact reflects 
the intended consequences of. individual project goals/objectives.) 
> Quest1on 1 ,3 , 

. bid proposal writers follow the new procedures as outlined 'in 
subsection 5.6 - Statement.,of .Impact in Guidelines and Procedures 
for Preparing FY 1982 Continuation Proposals by describing in the 
"Statement of Impact'Vthe target populations expected to be affected? 

Project Impact defines target audience (populations) as individuals, 

groups, or instltytions that could be affected by the activities of a 

• ? 

Research and Development program improvement project or program. 
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The guidelines request identification of the target population(s) 
expected to be affected as a result of the prograni improvement project. 
This implies that proposal writers, from the universe of all individuals, 
/ groups or institutions that could be affected by the activities of their 
proposed projects, are to choose and specify which groups can/will ^ 
reaTistlcally be affected. From this review, it is questionable whether 



this process^ if intended, took place. The results of the review, which 
follow, were based on whether or not a target audience (populttion) was 

r^^\ : . • 

defined, and whether proposal writers included the target audiences in 
the statement of impact that were subjects of goals/objectives and 
dissemination.. It does not addfes^whether the target populations were 
carefully selected, properly riarrowed, appropriate or realistic. 

Figure 3 suijimarizes the extent to which proposal writers identfft^d 
target populations In the statement of impact. Of the sixty proposals 
reviewed, 32 (53.3%) had defined target populatioTis. In 14,(23,3%) of 
the proposals not all populations tP be affected as described in project 
goals/objectives were Included in the statement of impact. The remaining 
four (6.7%) proposals in which the statement of impact was available for 
review did not identify a target population in the statement of impact; 
however, the target population(s) for the project was identified elsewhere 
in the^proposal . For 10 (16.7%) of the project proposals the statement of 



^-|m(^ot|Was not available for review. v . C ''•J 

In Appendix D the extent to which individual projects identified the 
target pppul a tlons in their .statement of impact is presented. 
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(Humb«r of Proposals) 



.e,de„tU1ed in .pact statements in flscawear me 
, oenree to which target populations are 
1^3!^ ^oStinoation proposals. - 



Qyesti5!Lli5. . Hiqsemination in' the 

. r.t.o.c.o„; . t.e pas. so^e p.opos. wHU. 

AS stated In the dissemination to 

^ ^ «^ Hi cqemi nation usuai'j " 

Have confused.impact and duse.. . ^^^^^^ 

4 rt Thus whatever outputs w 
^ synono.ous with impact. ^^^^^^^^^^^^ ^^^^^.^^ ,^^^,,„,,„„s 

,,.e disseminated by the prooe^ ^^^^^^^^^„ 

X- assumed to have;so.e impact . ^^^^ ^^^^^ 

although dissemination my be a part .^^^^^ 

,„paet. Effective d;iss™ination contnbutes to 

of a project. proposal components, 

the Smi^iSlJIliSSSSi^^rsUoLJ^^ ^ „S,,,emeht 

,,,„,st a "Oiss.in..on .annin, Out.ne ^ ^^^^^ ,,,^,„e 

r c Prniect proposals were r 
impact" in subsection S.e ^ 

the e^tent to which, the components wer 

i. ^i-ccpmination appearea n» 
to whit extent references .to d,sse..na ^ 
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Figure 4 provides the results of this review. Of the sixty proposals 
. reviewed, explicit references to dissemination appeared In 27 (4555) of the 
proposals' statements of impact, implicit references to dissemination 
'appeared In 13(2K6^) of the proposals 'statements of impact and no references 
to dissemination were made 1n 10 (16. 7X) of the proposals' statements of 
Impact.— ^Ten 06.7%) of the proposals did not contain a subsection on 
Impact; therefore these proposals* statements of impact were not available 
for review. 



Xeferences to dissemination, are 
explicit In the proposals' 
stateaients of Impact 



References to dissemination are 
lapMcU In the proposals* 
statements of Impact 



No references to dissemination 
appear In the proposals* 
statexents of Impact 



The proposals' statements of 
Impact were not available 
foe review 




(Number of, proposals) 



^^<f^rt i. Inclusion of references to dissemination in project proposals* statements of impact, 

V Considering that slightly less than one-half of the proposals' 

statements of Impact contained explicit references to dissemination, it 

is apparent that proposal writers may continue to mlsunderstan^d the 

relationship between dissem-ination and impact* Unless a project has no 

dissemination function, (the outcomes of the project goa1(s) require no 

dissemination), the Impact statement would logically require some 
> 

explicit references to dissemination. Examples of explicit references' 
to dissemination taken from proposals' statements of Impact Include 
the following: ' ^ v 




f "(The project*s) materi^rls will be disseminated to other Illinois 
community colleges, secondary schools and area vocational centers 
s.erving LEP students, and thus aid other instructors In the state 
to develop suitable vocational materials for this population," 

f "Teachers attending the workshops will develop skills which are 
necessary to integrate the microcomputer Into the existing 
curricula," 

f "Educators receiving training services from the project staff 
win acquire the necessary skills for operating an EBE program 
In their local educational agency," 

f "The dissemination and utilization of the results of research and 
project activities ^will eventuate in improved teacher training 
approaches which should ultimately change the way in which people 
are prepared to take their places in the labor force," 

V 

References in proposals' statements of impact in which dissemination 
was necessarily involved but not apparent, included such statements as 
the following: 

4 

f "Council efforts have resulted in Improved communications between 
area labor unions and school personnel," (implies dissemination 
of Information) 

§ "The impact of the project will be to provjxle a simple to administer 
« and use case stydy technique thlTt^cap be employed by the state or 
"local program s.ites to augment the information gathered through 
VEDS or the three phase state evaluation process," (Implies 
* dissemination of a mod6l ) 

• "(The) project makes available to Illinois schools and educators the 

entire list of V-TECS catalogs." 

• "The use of these materials is expected to increase student learning. 

Improve teacher performance, encourage teachers to develop better 
program structure and provide teachers' with an access .to useful 
instr uctionaT materiaTsT " ' ' , 

Noticeable in these implicit references to dissemination is the use 
by proposal writers of less definitive terms related to dissemination. 

Appendix E show^the types of. references to dissemination appearing 
In Individual project proposals' statements of Impact, 
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SUMMARY AND.RECOMHENDATrONS 

Summary 

The purpose of this review was to determine if changes in FY 1982 

^gutdellnes for .submitting proposalsTto the Illinois State Board of 

Education Department of Adult, Vocational and Technical Education, 
# 

Research and Development Section were effective in helping propos.al 
writers to Improve their statements of impact. More specifically it 

r 

was undertaken to determine to what extent proposal , wrt^li^s followed 
the guideline's specifications in preparing subsection 5.6 Statement 
of Impact in preparing their proposals. 

The Guidelines and Specifications for Preparing FY 1982 Continuation 
Proposals states; "The Illinois State Board of Education-reserves the 
right to reject any proposal received if It is determined that the 
proposal does not meet the specifications of these guidelines, % 

This review of the fiscal "year 1982 continuation proposals' 
subsection "Statement of Impact" and the subsections related to it 
Indicates a narrow majority of proposal writers followed the guideline's 
specifications for completing thisSsubsection, while a substantial 
number of proposal writers either experienced difficulty in following 
the guideline's specifications or failed to follow them by not including 
a subsection on Impact, 

The specifications in the guidelines relating to thpstatement of 
Impact for which the proposal writers showed the least concern was that 
of Identifying the "Statement of Impact^' as subsection 5,6, Slightly, 
over one third of proposal* writers gave attention to this detail, 

17 ' 



•The-failure of many proposal writers to designate this subsection" as 
"5.6 Stalement'of Impact" is certainly not a serious Infringement of 
the guidelines; however, the subsection was easier to locate when it 
was properly identified. 

The guidelines for preparing ])roposals require proposaU to be 
assembled in a sectional sequence which includes nine section headings. 
The section in which the impact statement is designated to appear, 
Section 5 - Narrative Description of the Project has seven subsections. 
It is important to note that for the proposals reviewed which contained 
a subsection on impact;, the subsection's location was sequentially 
correct in ninety percent of the proposals although the subsection 
may not have been labeled precisely according to the guidelines 
specifications. 

Relationship of intended. outcomes to impact. Proposal writers related 
the intended outcomes of project goals/objectives to the statement of 
Impact to some degree in all proposals that were available for review. 
In some proposals reviewed, the intended impact based on review of 
pro'ject goals/objectives implied the scope of impact to be within reach 
of the agency receiving funds' while the statement of impact described 
intended imgact on a much broader level. An example is a projeci^ whose 
goals/objectives relate to changes within an institution, a district or 
a region but whose impact^statement states or Implies the project will 
have impact on other similar target populations in the state or the 
nation while the plans for dissemination^ do not support this wider 
Impact. 



The extent *of dissemination of a project' output(s) wITl affect 
the degree of Impact whtch can realistically be expected to resultV 
from project activities. Proposal writers may need to give mor*e 
attention to the relationship of the statement of impact to goals/ 
objectives ^nd level of dissemination planned. If goals/objectives 
relate more to internal (intra-agency) change and if dissemination is 
to be iliternal rather than external, the amount of impact which can 
logically be assumed as a result ^project activities will be less 
than If external changes are planned anfl* dissemination strategies are 
at a level to -support the intended change. 

Identitication of target populations . A majority (53,3%) of the 
prQpos'a\ reviewed had defined target populations in their statements 
of impactXhost proposal writers identified intended impact for aV^J 
populations Wwhich their activities would be targeted. Often these 
populations were categorized as primary or ultimate target populations 
and secondary target audiences. Some proposal writers tended to be 
quite specific in identifying their targ^l^^pulations while others 
were rather vague. ' ' ' 

Based on the comparison of target population(s) identified in 
projects' goal s/objectjves and/or dissemination planning outlines 
with the target, populatlon(s) identified in the projects' statements 
of Impact, almost one-fourth {23.3%) of the- proposals reviewed did 
not identify one or more target populations in their impact statement 
which were identified in their goals/objectives and/or dissemination , 

Defined as Research and Development project activities that can be 
classified, into one of the following categories of: general infor- 
mation, products, practices, ideas and/or concepts. Assessing the 
Impact of Vocational Research and Development on Vocational Education 
' Programs Impact Assessment System Working Document. June, 1981 
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planning outline^ The remaining proposals. (6^72) did not identify 
a target populatiorv(s) in the impact statement, although a target 
populatlori(s) was identified in go^ls/objectives and/or *^the dissemi- 
, nation planning outline. Ten proposals (16J1J) did not have a state- ^ ^ 
ment of impact available for review. 

Most projects have more than one target population; however, some 
projects have designated only one or some of the populations for impact 
while working with other populations identified to achieve the Intended 
-impact. The Guidelines imply all target populations expected to be 
affected by the project should, be included In the Impact statement. When 
all target populations are not included^l assuming the target pQpulation(s) 
identified in goals/objectives and/or the dissemination planning outlioe 
to be the primary target, population(s) for impact could be risky. 

Inclusion of Dissemination. References in Statement of Impact . 
Two-thirds (66.6%) of the proposal writers included explicit or implicit 
references to dissemination in their impact statements. In most of 
these cases a dissemination . planning^ outline was also included in the 
proposal.; This would indicate the proposal writers recognize the 

1 

relationship between Impact and dissemination. However, If one looTcs 
'at the extent to which proposal writers described the outcomes of 
projects' goals/objectives in the statement of impact one finds only^ 
slightly over fifty .percent described most outcomes In their Impact 
statement. It may be possible' some proposal writers (the 26.6% who 
described only some outcomes) continue to address dissemination in 
their Impact statements while not giving appropriate attention to 
the Intended. consequences of the project. 
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. Recommendations 

A number of general and specific recommendations can.be made bSsed 
on this review of FY 1982 continuation proposals.' The recommendations 
are addressed to Project Impact Staff for consideration in revising the 
Impact assessment system arid for making recommendations to the-Illinois " 
State Board of Education, Department of Adult, Vocational 'and Technical 
Education, Research .and Development Section relating to revising the 
guidelines for completing the subsection on Impact. 

1. Stat^ent of Impact implfes one statement . Impact Inmost projects 
is •anticipated to be multifaceted and therefore requires multiple 
statements related to the intended consequences the project 
expec-ts to have on one or more target populations • The guidelines 
for the subsection im^ply that an impact statement should be 
written for each target population expected to he afi^ected which 
for most proposals, would be two or more. It is therefore recom- 
mended the title of this subsectfon be changed to Impact Statements, 
Intended Project Impacts, or Impact Objectives, ■ . 

2. Directior^s for' writing impact statements need to be clarified to 
have proposal writers produce an impact subsection of similar 
format which contains the required components . Two different 
formats are implied in the current directions with one of the 
formats suggesting a lengthy descriptive approach to defining 

• Impacts and one suggesting a short, concis'e description. The 
subsection directions begin *'the statement of Impact is a 
narrative description . . One might interpret this as "to 
tell or write about -in story form". Further^ the writer ^*s 
instructed that "in discussing the Intended consequences. . 
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(which implies several statements may be needed) two com'ponents 
are>equired to be included*. At the end of the subsection 
* directions an example ^impact statement is provided which is 
written In -a concise format much like a behavioral objective. It is: 
Students of teachers using the entrepreneur^hip education curriculum 
V." materials (target^ population) will develop entrepreneural skills 

(a consequence) and be able to apply these skills In their jobs and 
, everyday life (a consequence), The Statement of Impact subsection 
varlgd In length from-.a one sentence statement following the 
example impact statement format to several pages of narrative which 
may or may not have included intended consequences and/or target 
populations expected to be affected. If one or more statements 
such as the example provided is what is desired, the guidelines 
'must.be more explicit in stating^this. Following the example 
provided would make assessment of impact easier, both short term 
and in the future. 

3. Proposal wif^iters should continue to receive Inservice relating to 
defining the scope of a projectls impact. There appears to be a 
tendency for the statements of impact in some project proposals 

^ to Imply changes beyond those the projects^ goals/objectives and^ 
dissemination plans would seem to make pos$1ble. - , 

4, Proposal writers should continue to receive inservice relating to 
defining the target populations to be affected by the project .' • ^ 
The degree of specification desired in identifying target audiences 
for a project* s Impact needs to be clarified*. The choloie. of a 
target populat1on(s)*also needs to be appropriate and realistic. 
When target populations are quantified, some proportion or percentage 

^ of the total population to be. affected should be estimated. 

i 



5, Reviewing Statement of Impact carefully as it relates to other 
sections and subsections of a proposal should be part of the - 
evaluation review^of proposals for fundi-nq . Those who will 
serve as proposal reviewers need additional technical assistance 
In this ar6a, 

6, The order in which Statement of Impact occurs in Section 5, ^ 
Narrative Description of the Project should be changed . The 
Statement of Impact should be loca^jted closer to the subsections 

to which it is related (i.e. introduction, objectives^ procedures),. 

Ideally, it should be part of subsection 5,2, objectives. In this 
case subsection 5,2 would contain general objectives and impact 
objectives . However, if it is to, remain a separate subsection, 
it should' follow subsection 5;2h Currently it follows the sub- 
section on .dissemination and both follow the subsection op evaluation. 
This Is not a logical sequence of the' components in section 5 and the 
sequence 5-hould be reconsidered. , , - 

7, Reviews of final reports for achievement of short term impact based 
on the statement of impact should be^ completed at the end of each 
funding phase, Unl^s this is done, the benefits of specifying 
Impact will be weakened or lost,* and the appropriateness of future 



I, 



impact statements may be affected. 

The^al^ve recommendatlonsf if implemented may serve to improve' the 
process of planning for impact and Increase the actual' impact which will- 



be realized as a result of prpgram improvement projects. 
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FY 1982 PROJECT PROPOSALS REVIEWED fOR 
TREATMENT OF SUBSECTION ON. IMPACT 



Project 

Proposal 

Number 



TitTe^of Project 



.1. 

2. 
3. 
4. 

5 ' 

6. 
7. 
8. 
9. 

10. 

n. 

12. 

13. 

14. 
15, 
16. 

17. 
18. 



ERIC 



Tr1-County Industry-Education-Labor Project 
minots Vocational Curriculum Center. 

Relationships Between, Vocational Education and Job Creatlpn, Phase III 

Establishing a Hotel /Restaurant Management Degree and Culinary Arts 
Program, • • ^ . ^ ' 

Continuation of a Longitudinal Study of the Nontraditlonal and Social 
Impact of , Vocational Education on Individuals in Illinois 

Illinois V-Tecs Study, Phase IV 

Truman Bilingual Vocational Center-Project Mainstream 

Women's Pre-Apprenticeship Technical Assistance Project 

Ethanol Spark Ignition Engine Conversion Dissemination of Technical 
Achievement and Curriculum Development 

Comprehensive Promotional Program for VocationaJ Education at the 
Local Level 

A Computer Assisted instruction Program 'Model for Occupational 
Education Leadership and Administration 

A Plam to Develop and Compare Two Vocational Education Prograras for 
LEP (Limited English Proficiency) Students 

Surmounting- Architectural Barriers, to the Handicapped in Vocational 
Education, Phase III 

Microcomputer Skills: Curriculum Development and DTssemination 
Curriculum Publications ^earinghouse 

Improving Services to Disadvantaged and Handlsapped Students In 
Vocational Education , - ^ . . * 

Rural Health Occiipations Projects 

State Guide for Industrial Education, Kindergarten thru Adjult 
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Project 

Proposal 

Number 



Title of Project 



19. 

20. 

21. 
22. 
23. 

24. 

25. 

26. 
27/ 

■28. 
y29. 
30. 

31. 

32. 
33. 
34. 
35. 

. 36. 
37. 

t 38. 
O 

?ERIC . 



Illinois FY 1 982^ Participation in the ConsoVtium of States for the 
Development of Professional Materials for Vocational Education 

Technlcal^ssistance and Dissemination Network: Illinois Special 
Needs PopuTatlons ' ' ... 

Methods and Materials for Entreprene'urship Education, Phase V 

Marketing/Distributive Education Curriculum Guides 

Research, Evaluation and Program Improvement for Limited Eng^lish 
Proficiency Students in Vocational Education 

IVA-DAVTE (Illinois Vocational Association, Department of Adult, 
Vocational and Technical Education) Occupational Education Mini 
Workshop " 

Development of Multi-Cultural Competency Based Vocational/Technical 
Curricula 

Illinois Core Curriculum in- Agricul ture 

Curriculum 'Development and Implementation of Alcohol/Ethanol Training 
Program 

Adult Re-Entry and Work Transition Program, Phasfe III 

A Career Program for Vocational Teachers ^ 

Principals' Practicum in Vocational Administration and the Assessment 
of Training Needs of Vocational Administrators 




Job Creation-Dissemination of Instructional 14ater1als ^and Activities * 
for Vocational Education Teachers 

Irffusing Experience-Based Education in Illinois 

A Developmental Unit for Maximizing Training Level Curriculum Offerings 
Project Access 

Experience-Based Education, Appal achia Education Laboratory Program 
for Special Needs- Students ^ 

Institute for Native American Development. 

Demonstration Center CIVE' (Competency-Based Individualized Vocational 
Education) Project ^ • 

Recruitment, Retention and Retraining of Vocational and Technical 
Teachers in Illinois • ' 
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Project 

Proposal 

Number 



39. 

4o: 

41. 

4Z. 
43. 

44. 
45. 
46. 

47. 
48. 
49. 
'50. 
51. 
52. 

53. 
54. 
55. 

56. 
57. 
58. 
59. 
60. 



Title of Project 



Ac Determination of Best Practices for Incorporating Basic Skills _ 
Training In Vocational Education in Illinois *• 

Upgrading of Locally-Directed Evaluation 

Development of a Comprehensive-, Sequential -Vocational Home Economics 
Curriculum, Phase III ' 

Sex Equity Cadre " ' • 

Developmejit of a Pres.ervice Training Program for Vocational Teachers 
of LES flimitpd English Speaking) 

Bilingual Vocational Educa.tiori Project 

Programmatic Emphasis in Vocational Needs Education 

Development of an Information Base and Exchange System for the 
Improvement of Industrial Oriented Programs In Illinois, Phase III 

Inservlce Training for DAVTE State Staff Personnel 

East -Central Currlculum Coordination Center 

Utilization of Computerized Career Information Program "Career Spectrum" 

Revltallzation of Orbah Education for Urban Youth 
i 

The Vocational Instructor Consortium and New Teacher Recruitment Project 

Occupational Coordinator: Southern Illinois University, Charleston 
DAVTE Liaison Officer 

Occupational Education Coordln^ition Chicago State University 

Occupational Education Coordinator Illinois State University 

Occupational Teacher Education Development. DAVTE liaison, Southern 
Illinois University, Edwardsville ^ ^ 

University Occupational Education Coordinators Eastern Illinois Univ. 

Occupational Education Coordinator Western Illinois University 

Occupational Education Coordinator University of Illinois 

University Occupational Education Coordinator Northern Illinois Univ. 

Occupational Education Project Governors State University 



ERLC 
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, , Appendix A-2 

FY 1982 PROJECT PROPOSALS NOT AVAILABLE FOR 
REVIEW OF TREATMENT OF SUBSECTION ON IMPACT 

' 1. Review and Revision of Occupational Education Curriculum and Course 
Master List . ^ • " 

2. Exemplary Home Economics Project 

•*3. V-TECS 

4. Articulated Marketing Program 'Curriculum and School 

5. Vocational Education Slide Tapes 

6. PREP Statewide Conference 

7. Industry, School and Community-Based Voc. Ed. 

8. High School and Beyond ^ 

9. Administrative Leadership Fellowsiiip in Vocational Education 

♦There are two V-TECS proposals, the one included here beavs the funding 
agreement number R 33 52 X 0541-162 



Appendix B 
TkbTel ' 



^ Identifiers Used for Identifying Subsection on Impact 
for Individual Fiscal Year 1982 Continuation Proposals^ 



Identifiers Used for Identifying 
Proposal Subsection on Impact 


Individqal Project Proposal 
Numbers f§r Which the 
Identifier yas Used 


Total 


Statement of impact identi^fied 
as subsection 5,6 

V 


1» 8^ 10, 11 , 13, 14, 18, 
19, 22, 24, 25, 26, 28, 
'29, 30, 31 , 32, 34, 36, 
37, 43, 51 


22 


statement of impact identified 
as subsection 5,5 


alt: 

4, 38, 39, 4:1 , 52 


5 


Statement of impact identified 

by a number (s) other than 5.5 or 5,6 


2, 7, 46, 58 


4* 

• 


Statement of impact identified 
by a topical heading only 


5, 9, 12, 15, 16, 17, 
20, 23, 27, 33, 35, 
42, 44, 45, 50, 56, 
57, 59 


18 


'Statement of impact identified by 
a letter in the alphabet 


6-- 


1 


Statement of impact subjection 
not available for review ^ 


3^ 21 , 40, 47, 48, 49, . 
53, 54, 55, 60 


10 






60 



Note: Refer to Appendix A-1 to ident-ify project titles for project numbers 
appearing in the abpve table. 
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Table 2 



Extent to- which the Statement of Impact Refjects 
Intended Consequences of Project Goals/Objectives 
for Indiv4dua;j!^FY 1982 Continuation Proposals 



Extent to which statement of impact 
reflects intended consequences of 
project goals/objectives 


indiviuua i project proposa is 
reflecting degree of intended 
consequences in column 1 


Total 


Statement of Impact describes most 
outcomes of project goals/ 
objectives 


o A c c o Q in 10 10 
2,4,5, 6,- 8, 9, 10, 1^, 13, 

15, 16, 17, 18, 19, 2-2, 23, 

24, 25, 26, 28, 29, 30, 31,32, 

34, 35, 42, 43, 44, 45, 46, . 

CI CO c c * 

51 , 52, 55 


34 


Statement of Impact describes some 
outcomes of project goals/ 
objectives 


1 . 7, 11 , 14, 20, 27,' 33, 36, 
37, 38, 39, 41, 50, 57, 58, 
59 




Statement of Impact does not 
describe outcomes of the project 
goals/objectives 


none 


0 


Statement of^ Impact not' available 
for Tevlew 


3f 21 , 40,-47, 58, 49, 53," 54, 
55-, 60 • 


10. 






60 - 



Note: Refer to Appendix A-1 to identify project titles for project numbecs 
appearing in above table. 
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Appendix^O 
Table 3 



Extent to Which -Target Populations Arje Identified in 
^ Statement of Impact in FY 19§2- Continuation Proposals 


• 


Extent of Identification of 
Target Popul'Stions 


Individual Project 
, Proposal Numbers 


Total 


Target populations to-be affected 
are identified in Statement of 
Impact 


1, 2, 4, 5", 6, 7, 8, 9, 10, 11,. 
12,- 13, 16, 17,. 19, 22, 23^-^24, 
26, 28, 30, 32, 34, 35, 36, 42, 
43, 44, 45, 46, 51, 52 


32 


Some target populations to be 
affected are identified in 
Statement of Impact 


14, 15, 18, 20, 25, 27, 31, 
33, 37, 41, 50, 56, €7, 58 


14 


The Statement of Impact does not 
identify a target population 


29, 38| 39, 59 


4 


Statement of Impact, not available 
for review 


3, 21, 40, 47, 48, 49, 53, 
54, 55, 60 


10 






60 



Note: Refer to appendix A 1 to identify project titles for project numbers 
appearing in above table. 
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Ta.bl e 4 



Types of References to Dissemination Appearing 
in Individual Project Proposals' Statements of Impact 



Type of Reference to. Dissemination 
in Statement of Impact 


Individual Project 
• ^ Proposal Numbers 


Total 


References to dissemination are 
expli£,it in the proposals' 
statement of impact. 


2,-4, 7, 8, 9, 10, Tl, 12, 14, 
18, 20, 22, 23, 24, 25, 27, 28, 
30, 31, 32^ 34, 35, 36, 43, 44 
45, 52 


27 


References to dissemination are 
implicit in the proposals' 
statement of impact 


1 , 5, 6, 15, 16, 17, 19, 26, 38, 
46, 50, 56, 57 


13 


No references to dissemin^lon 
appear in the proposals' 
statement of Impact. 


13, 29, 33, 37, 39, 41 , 42, 51 , 
58, 59 


10 


The projects' statement of impact 
are no* available for review 


3, 21 , 40, 47, 48, 49, 53, 54, . 
55, 60 - , ' 




>j . 




60 



Note: Refer to Appendix A 1 to identify project titles for project numbers 
appearing in above table. 

r 
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Appendix B 
Summary of AERA Conference Attendance 



\ 
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Dr, Earl Russell, as Project Director of the two projects, "Assessing 
the Impact of Vocational Education Research and Development on Vocational 
Education Programs'* and "Vocational Education Program Improvement Dissemina- 
tion," attended the American Educational Research Association Annual Meeting. 
This meeting was held in New York from March 1^24, 1982, At the meeting, 
Dr,. Russell attended a substantial number of programs which reinforced the 
objectives of the Impact Assessment and Dissemination Projects, Additionally 
attendance at the meeting provided an opportunity to discuss and obtain sug- 
\gestions from other researchers on the refinement of the Illinois Dissemina- 
tion and Impact Assessment Systems in the 'final year of these two projects. 
Some of the sessions attended and papers collected incliTde: 

. "Evaluability Assessment of the Adult Education Program (AEP): The Results 
and Their Use," Presented by Darlene Russ-Eft, 

"Participation of Women and Minorities in Vocational Education," Presented 
by Mary A, Golladay, 

"Following Vocational' Education Students Through National Longitudinal 
Studies," Presentation of tables by Jeffrey A. Owings, ^ 

"Illuminative Evaluation of a Second Year Medical School Program," Presented 
by Marcia Z, Wile, 

"Preparing Facilitators for Implementation: Mirroring the School Improvement 
Process," Presented by David P, Crandall, 

"Two Large-Scale Studies - Dissemination and School Improvement," Presented 
by David P-. Crandall . . ^ ' \ 



"Assistance and Enforcement as Strategies for Knowledge Transfer and Program 
Reform," Presented by William A, Firestone. 

"A Diffusion of Innovation Based Model of Instruction: Attributes of Innova- 
tions as Predictors of Classroom Learning," Presented by Scott M, Elliot, 



"Networks for Innovation: Characteristics and Dynamics, with Examples from 
Education." Presented by L, Allen Parker. 

"Discontinuation of Innovative Programs," Presented by Ralph Parish, 

"Developing and Refining Linkage Know-How: Some Methodological and Conceptual 
Issues," Presented by William R, Thayer. 




